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I.,T30WC2I03! 
?he Problem 



ouppose there is a set of objects, U, 2, ...,:}, onfi a set 
of tests, iri t T2 f . . . f ?Sl . tfhen a test is applied to an object, 
the result is either T or ?. Assuae the tests may vary In 
cost and the objects nay vary In probability of occurence* Oae 
then hopes th»it an unknown object say be identified by apply- 
ing a sequer.ee of tests. The appropriate test at any point 
in the sequence in general should depend on the results of 
previous zests. 

The problem is to construct a good test scheme using the 
test costs, the probabilities of occurence, sad a table of 
teat outcomes. 

To pux this in concrete terms, suppose an induction cen- 
ter wishes to deploy recruits into Jobs appropriate to their 
e salifications. Jurther, suppose all ass are kaowa to fit 
oae of rive doos: general, rifleman, typist, pilot, or spy. 
Possible tests aight be: 

I.Q, : >100-*T 
Vision j normal-* T 
dexterity: good—* ? 



She cost-effectiveness oriented sar^ent constructs a table o: 
qualifications such as shomi in figure 3 (in general I 
call such tables, tables of outcomes). He then sa^es a lisi, 
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Figure 1. A table of autcogss. 



vision normal? 
T 




dexterity s°oa? 



Pigure 2* A ocssi^le test tras. 



of zc.$z ccjts an3 r, lict of f*rafri.billti&D cf ;ob ty?es. 
TlMlly ho Att&£?is to iivcnt c coot tost tree. 71-ura 2 j£c- 
turas one tree he xijht select. 

A ziorz serious possibility ;;oulO involve construction of 
a lla^ools tree for n computerised doctor. Othor apflic&tio::c 
of c^od trse building methods are cot obvious. : ;-.oje to 
employ soae of the ideas that hr.ve evolved In a prosr^E th^t 
searches semantic acdories of the sort described by tiuilliai:, (D 



Exhaustive ^numeration 



One's first reaction might be to try all possible trees 
and select the best one. 2his, hovrever, is impractical for even 
moderately lar^e problens since the nuisber of possible trees 
grows rapidly with the auraber of objects and tests: 

Let the nunber of available tests be N. Io simplify the 
calculation, we will consider only those trees of the form shown 
in figure 3a. In these trees any path descends dova throurii i 




uniform—counted 




erratic — not cou^tad 



?l£ure 3. ?rea forzs. 



tests, where that depth C ir- Jiwt *x»££i3%a;.% to U--rc iz 
leest ^s BDfiy ter:Qlr.nl tamflh*a as object?. Oar nuafeer will 
be a lo;.*or bound, ciuca trees that branch erratically as t;.at 
of fljurc 3b will not be counted. 

Phe nuabar of possibilities for the first test will be :;. 
The number of possibilities for the second test Will be ;;-!, 
but Kfl now have two tests to be fixed • For each of the .c-1 
ways the first of the two is fixed, there arc :;-l ways to fix 
the second* Hence the total number of variations so far Is 
n-(;i-l) ■ The number of tests now left unused at asy tree 
tip Is K-2 and there are 2 choices to be made- The nuaber 
of variations is therefore increased by a third level of tesxs 
to N*(;>1) 21 *U"-2}2 2 . attending to d levels we have 
3S*{H-1) 2 -. .(S-fd-1)) 2 ^" 1 - This nusber rapidly becomes uncoa- 
forta&U., lot exaapUi a sarapie probles worked in the 523UIT5 
section involves only 5 tests and 3 objects, d is then 3 

yielding; 

■ 

number of possible trees > 5 • 4 2 • 3^ = 6,430 . 

In a seoond sample problem, there are 7 tests and 16 object3. 
In this cs.se t d is 4 and the lover bound is; 

number of possible trees > 7 * o 2 * 5 4 * 4 3 - i o* ID 10 

:revious Tfort ( 

In 1954 Sla^le wrote a paper on this problem, but he 

VU concerned only with the special case in which objects be- 
long to one of two categories, aod the ooal is only to decide 
to which ca^e^ory an unknown object belongs, other restric- 



tlono or. t;,c ftatsttre of aha v*blo of outoax&c further lfc*.lt 
his discussion. J 

;. later r*apcr by SolliioJ:"' offered sji ele^eirtary haurlctic 
aj ownch, but :;e averfllaplified the problem by G3susiir; 
r.ll -cects to bo of equal coot and all ob;*ect3 to bo of equal 
probability. 

His Idea 1b simply to insert at each point that tost vhich 
most nearly divides the object set Into two groups of equal 
probability. 

Finally, Beirraald and Solandf*) present an interesting ae» 
thod that at first glance see»ns to lead directly to an optical 
solution of these test tree problems. On closer ei^Qi*atlon, 
however, It may be seen that their scheme is only slightly 
better than exhaustive enumeration for the class of problem 
of Interest hare, This ifi shown in the appendix, 

A Heuristic Abroach 



Since no entirely satisfactory solution could be found in 
the literature, I developed a heurisxic XIS? prograa that seezs 
to do a reasonable Job on a number of test casec The prosraz, 
as described in the next t?ro sections, exhibits several concepts 
found in intelligent programs: there is s simple fern of plan- 
ning; -here is a focusing of attention; and there ere a number 
of local heuristic operations. Tha length of the program 
was about 12 pages. 



Sea £4) for details. 



?©*jj?l i ■ fLi* action 

?>.\ ta&io i<ic : of t':o psdeeui is tg taffsazoClj c£.*^*d t i2 
improve o parti-lly forced t:cu of te-tc, .\ function nacafi 
TQSPSLld is ia charge of local OiWatldlU :mfl trieii to Apply 
heuristic transforiantlor^ at tbet r-rt of the tree to vfaiah it 
is directed by a higher level function n&scd 2XBC0TIVE* Ths 
heuristic resources of F033US -re described ia the followins 
paragraphs. It should be *:oted thai only the essentials or 
the functions are indicated, with no attention to the trouble 
the actual procraras face in handling normal variants of the 
situations described as wall as the aany unusual cases. 

3uSdiaS 

Suppose the SrCSOUriVE directs POaEKAS to work at the spot 

Indicated in figure A- Since a timber of objects have yet to 
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?lxure 4. The use of 3JU3. 

be sapzratad, it is desirable to pick suitaila zests fro- 
a list of those not previously used and ;raft theTi on-o th< 
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tr^c. *hL« Is IvtBdlOi! by l;>^ is3j; facet io&, .:ot it* Iz o*G*j*y 
3cBirafcl9 Kii\t Bull celect r. low cost tcot. Or* th© other h:-.v» H 

if r._: aoat* wcya the swao* it would bo a c^oi Idea to DQicet 

the tost that bed divides the objftctfi Into g^ttl-ptfobsfclo craujifi. 
If :;or^in;T out a few examples is not convincing enough, further 
justification for this nethod of selection may be found in 
(Jallns*** '■ essentially the sane problem is diccussed in the 
contest of ttinirnizlnr- the average length of the codewords in 
a coisxur.icatlons system. 

But since the test costs do vary, a compromise ie neces- 
sary . The test actually used is the ona wMoh naxicises the 
following function of cost and splitting: 

Quality (test) = Uzjjlll 

5- 

vrnere ?2»?t a7e the total probabilities of objects sent 

- dewa the right sad left branches 
£.Ed T is the test cost. 

■ 
If the function evaluates to the sane number for two different 

tests, the BUD function is said to be indifferent to which of 

the tifO is selected- Figure 5 shows n indifference curves" for 

the function and offers some insight into how the scheme works** 

Figure S shows two other possibilities for tJia quality function, 

but the curves of figure 5 gave the ooapronise intuitively 

desired, and indeed the others proved inferior experimentally. 

This is admittedly flimsy Justification, but intuition was 

exercised In order to expedite setting a working program- This 

will be seen again and the reader should realise that the tttdt- 



t> These curves are reminiscent of potential lines, and indeed, 
one aay think of the process as selecting the tect v;:zh fct£Swst 
"potential." 
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Figure 5. Lines of indifference. 
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Pi Sure 6. Lines or infiiff srer*ce for 
other quality funotloac. 



■:c:.'.-i ;>rr r.<atos3 which ~"c-r "crc set Lhriuj-: iatrt&tlvo ;u-:;-.- 

- 

.- i*fr--Mtut~ ,1- 

It is ajvsy to see no.; *ho £uJ3 function can av;e .:iMt:u:co. 
Suppose, for example, that throe tests are av&liable for £rtft- 
las tit sonie end point In a partial tree. Suppose further 
that they Jivida the probability as indicated somewhat abstractly 
by figure 7. BUD would obviously rflco:aer*d that test Tl be 
followed by test T2 to produce the division of probability 
shown la figure Sa. But with only a slight increase la cost, 
one could cet much superior overall probability splitting by 
using 13 followed by T2 as sbown in figure 3b. 

Of course one could merely complicate BUD by ezaaing 2-level 
combinations of tests ia ftbout the ssae wy, but the aucber 
of combinations increases sore than linearly with she number 
of tests and is to be avoided. Instead, this program uses a 
function naaed 3C23TITCT3 to loo* at a particular point it the 
tree with a view toward changing the test. The test is changed 
if another gives a tetter quartering of the probability with the 
sane subsequent tests. Hence SUBSTITUTE would fix the error 
made by BCD in the current exanple i>y the change shown la figure 9, 

SU33TITUT2 
> T3 





Figure 5. Change effected by S^^JSTITUTS 
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Figure 7. Three available tests, 
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?isure 8. Altsmstive two- test partitions. 



A*.t4i d- - *-'~i> •• ooaproslse v.-lt:. oost io f.t«»tr; -.v*. • •- 
j.-...a L -c is xr4« uuIcod r Atffcraat tent ylfeUa •- jc—ttr .-.- ;b ' 
LA 

4 

is which the parameters and rationale ere anc-lo^ouc to taoao 

discussed under Budding . 

XSturally this idea could be extended to still higher 
orders, but I doubt if any significant caln would be hcd. 

Suppose now that one is examining e tree tip of the fora 
in figure 10a. One wonders if perhaps & little alteration 
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]3] to? U3 10} 

?igure 10* A possible exchange. 



would be better, say to the forn shown in figure 10b. C fi l- 
eulatlflg expected costs it is clear that the chango should be 

cade if 

3 r; . = OOSt of test TX, p = P( \ A 3 C 3). 
cr.ii 



& 






Gaacrr.ll-jcu a bit, this Idee enn bfi applied ;^^ViCre In t&* 
tree end us such, constitutor the ^Z3rM*;;c function. Cne 
nodi Dai/ think of A, B and J in the abave paragraphs zu 2££o 
oJT objects instead of simple objects- ;;o;: t ho;;evor, the? func- 
tion sust be prepared to do some siore extorsive .fork on the 
tree if subsequent tests are already in place. Pl^ure 11 helps 
indicate i;hat should be done, notice that the objects of 
set C remain together in the new structure. So do those of 5. 
One expects that the subtrees used to divide up sots 3 and 
are still good and should simply be moved to new locations, as 
in figure 12.* , 

The trouble is that some of the objects of set A likely 
lavaie the branch of set 3- Ifow 3ZGHASG3 is not necessarily 
a good idea and errors are nade. For this reason, the swapping 
condition was altered so that swapping does not occur if possible 
rains are slight. This is dona through the introduction of a 
paraaeter <*: 

. Oil * ?( [3] ) >*C ?2 * P( U] )-h «»P- 

<< uas set nt 1-25 with seemingly satisfactory results, 

■ 
Sveettlnn 



Oonsider the situation of figure 13a. Svreep will transform 
the tree to the form shown ia figure 13b if Cm 2 < 0m* So ianedi- 
ate change in expected cost results, but it see:25 a cood idea 
to "sweep 11 longer tests dOWfl the tree structure, for it zar 
well be that they can be eliminated in scse of the sebseruent 
paths and there result in an eventual savins. 
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Plgura 1 1 . A senerr.li.23d exchange. 
A ( 3 and C are sets* 



subtree a -* 




SXCHAEGS 



subtree 3rj 



subtree C 




subtree A***' \ 



subtree Z 
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fclgu.ro 12. "odif io^tlons ap^roori^tj for 
Generalized application of SX33A33E. 
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Figure J 5. A sweep. 
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?iRure 14. A aoro carcipllcfttod sweep. 
A| B» 0, D, X nnd X are sets. 



-I? 

^i&c 3X3KA*2£| the fTi*S£ function In uc* 1^ a c«»rr,l> 
r- - -i-r.. I- i;ill taie i;.c tree of fi;; 11 *** !'- ar *- »TES3fora It 
■to t3:*it of finiro 1** *- oert&in coaSiyioco aj*e ^ct. j*or o»e 
thins Oj >fc ac bofore. Lut filao notice that O&l? the sets A 
cad 3 are preserved; C and 3 Will be in general divided between 
>; and Y. In order to avoid a complete m&ucllilSj the function 
refuse^ to effect the change unless D notches X fairly veil, 
and C matches Y fairly vrell (or if C with X and a with y). 
''Match fairly well*' means 65$ or acre of the objects in the 
first set are found in the second. 

3VJ23? Day also bo used even if 0^< 0^ if 0^ >> ^ and 
the other conditions hold. This second criterion was implemented 
since in such cases the elimination of T3 seems lifce a soog deal 
( >> ae&as asrs than 3 tines here). 

Naturally S'r;S2? also looks at the tree with the above roles 
of T2 and T3 reversed. 



i 



Pruning 

ZXSEA^GE and SK32? cause aany objects to course down 
different paths. The result is that "dead" tests, or tests thai 
c^use no splitting, say be found here and there in the tr=e. 
Xhe-a ?033KAS encounters such a test, it uses P3HEB to eliminate 
it as in figure 15. SHONE is repeated if the test structure it 
elevates is also headed by a dead test. 
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PIrtire 15. ?he use of 23USB. 






Lvo ?r;actio^ 



,;ith tile arcaaentc rlura of heuristic operations Just 
levl^veS. ono csuld avaoalAs a siaplo supervisory procrao- 
Iho prolan could siu-ly vork its Way ftoim the mowing trse 
by c%llir*c itcelf recursively on the branches it generates. 

Notice that a substitution at one point might vrell set up 
the higher test point for a sweep or other change. But In 
the simple scheme mentioned above, there 3s no way for the 
program to back up and review vhon such changes marte reconsi^ 
derations of a higher level choice a good idea. This consi- 
deration led to the development of 3XEO021VB, a somewhat sore 
elegant supervisory progras. It may be helpful to refer fre- 
quently to the flow ohart of figure 16, as the prolan is 
explained in the following paragraphs, 

3oals of EX30UgI73 

The Job of 5X3C35XY3 is to focus the program's attention 
at an appropriate point in the partially constructed tree- In 
diagrams the convention Kill be to indicate the center of atten- 
tion by a saall arrow (-*)• 5*wo tests immediately following a 
given test are called its daughters. ?ho given test is then 
referred to as the parent of its daughters. 

SMZCUTIVS must bs able to do sore than Just back up and re* 
vieu an old choice* To avoid wasted effort it should not >:ori 
on a test point if a parent of that test is to be reviewed, 
because a change may in fact con^letely change the nature 
of subsequent problems. In figure 17, for example the problem 
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?isurs 16. 3X331TCIVS. 
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Figure 17- Execution of parent problea before daughter problsa. 

of point 2 has a different set of objects to consider as a result 
of a higher level operation at point 1. s;C30U?IV3 should simi- 
larly avoid deep extension of one tree branch while neglectins 
another. Otherwise, a reviev and subsequent alteration generated 
by a later look at the neglected branch might well render nost 
of the vori useless. See figure 13- Finally, EXBOTTXVE should 
have soae kind of cycling protection so that chains of circu- 
lar operations are not repeated indefinitely. 

Before explaining how these things are done, the role of 

FQ32MAH ''ill be disou3sed- 



Tna Zorenan 



Once BXBQ3IIV3 has picked a center of attention, 7G3&A&& 
cleans up the area v;lth EUJE3 and, if necessary, extends the 
^ree %-ith 3UD; it tries to apply SC332I2UT3, EX0EA553, or 3533? 
in turn; and finally it use3 JSUE3 and BUD in oasa a transfer- 
nation has left sotj3 ^arba^e in the in-ediate tras struc-ure. 



:,si£ tha* tlla *** tent~avl$oii" $z o;QT.\tlo^z o^n be 1155 i*C 
only ii* the teat at the center of attrition ;,:,:> daichtsre- 
in each of it3 tVO branches. A flo*r chart fcr 70;^;^.:: Is la 
figure :?. 

?hc rro'oloa list; ?!■' ar.ln- 

Once EX3OTIXV3 has piclcod an initial test, it places a jacket 
of Information called a problea on a list oalled the pro bier: list. 
A problem conalots of 1) a pointer to the test it is associated 
with In the tree f 2) a nunber indicating its priority, and 
3) other lists that aid the operation of PQHBHtfi but r-re of 
no concern here, is can be seen in the flov; chart of figure 16, 
3X3CXRTXYB enters a loop around uhich it circles until the 
tree is complete* 

The first step in the loop Is to apply FOEEHAS to the first 
eatry {the one with highest priority) in the problem list. 
Of couse, vhen entering for the first time, there Is only one 
entry. 

If 70B2&LS only extends the tree with 3UD, or does nothing, 
SSSOUHrra siasply creates new problems associated with any 
daughter tests otherwise without associated problems. It 
then removes the current problem froa consideration by placing 
It on a list of old problens, the recall lisfr, The use of the 
reoall list rill be explained later. 

?he new problems are inserted into the problem list 
according to a priority, vhich is simply the number of objects 
that terminate paths fron^fssoclated test. See figure 20 for zo-e 

*** Insert 3 J35? and S'Jt??"-:JTS in this position. 
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?i 5 ura 19. POaSSOH. 
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?lo^re 20. Priorities and the movement of attention. 

examples . 

In establishing this priority, the program nay bo said to 
exercise a simple sort of planning* 

This priority scheme assures that a problem Hill net be 
considered if a problem associated Kith an ancestor is also 
on tha probleu list. This is true since any test in the 
tree certainly has more objeots coursing through it than have 
any c£ its progeny. 

The priority defining scheme also helps avoid extensive 
crotfth of one branch with neglect of others. Shis follows 
since, in general, the repeated partitioning of the objects 
rapidly reduces the priorities of problems further down the tre 
See figure 20 for an illustration. 

Finally, notice that, as In fisure 20, the daughter in tht 
branch vith nore objects is attached before the ether. Shis 



tzrji like a z^l tdiw tcpwe a ok«CC *- : TiX«rtio:i '-.-; M 
i .>..;: level ;rcV.c.: i: so?> li\cly to cs.^irro the -o, c 
*:o~al?-tcd o* t ; o liiu^htcr brr.achcc. ;-."«:*•: £fmg tl-a 3«ar ir: nor* 4 
* y then toe UMttGd. Tjj1« aovoaoat of £t£estiOa fcaci: u; V:.'j 
tree aeens r.oro Llirely xroa a highly ?opulctefi fcranoh ifltsasfe 
more variations or object deployment ^rc possible. f 2hl$ yi&lds 
more values Tor the quality functions, increasing the chance 
that an improvement any be found* 

The Hecall List 

Saturnine to the top of the loop, notice that if TOEBffiUZ 
does in fact chance a test by exercising 2UOHi-SS3 t SU£STI2U3j 
or SKSBFj then the program woris throush a different set of oper- 
ations. ?irst f the auxllliary lists carried in problems iniioato 
what tasts have previously been used and vbat objects are to 
be handled. The test-svrapping heuristics frequently charge 
the course of objects through the tree struoture and invalidate 
the auxllliary lists. Trouble is avoided simply by destroying 
s.ny problems found subsequent to the change. 3ear in nind that 
only problems are eliminated, not the tree structure itself. 

The parent of a changed test should be re-exanined sinoe 
change in a daughter cay la zurn call for a transforation of 
the parent. Kotice that the parent problea resides on the 
recall list. To initiate review, it is nerely noved froa the 
recall list to the problea list at the spot appropriate to its 
priority number. 

Figure 21 illustrates these steps. 
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3^^ assoetatafl ?rol>lesi is on tfto ric.'ll list 
?~*Spaooiitrt nrobiea if on t;i3 .oroolas liist 
>■ ladiccCOj oor.«.T o;" nttsiitia:: 



3UD 
3UD 





Figure 21. Hecall of an old problem. 

Qygllafl 

2he cycle protection sear is not included on the flow char: 
to avoid cluttering, The current scheme xerely counts the 
number of times & problem has been reviewed and, if nore than 
twice, further change is not allowed. 



lamination 

Che problem list will be e^-ptj when the tree is fully grown 
and all necessary decisions have boen reviewed- SXSOUriVZ then 
exits from the loop and announces that it is finished* 



Tlio completed pro^ma ::as r.r^Ilcd to 10 situations of :;fclc; 
two are reviewed i£ tiis section, She *Ir^t SG3 tl-.o t&vlc af 
outcomes* probabilities, and oasts Bfrotffl in ts.ble I. Notice 



objects ' 


A 


3 


3 


D 


^ 


? 




H 


costs 


ests-y 

71 


T 


T 


T 


T 


7 


? 


"' 


7 


1.1 


£2 


T 


T 


P 


P 


T 


T 


P 




1.2 


T3 


T 


P 


T 


71 


T 


7 


I 




1.65 


14 


T 


T 


p 


T 


T 


T 


P 


P 


.93 


T5 


? 


P 


T 


j 


? 


P 


T 


7 


1.55 



(all probabilities e^ual .125! 



Table 



that all probabilities are the same and that tfra first thraa 
tasts ell divide the objects into ea.ually prabsble groups* 
?he regaining parameters ware fixed more or less arbitrarily. 
Ifte net result is a data set on which 3M23UTI'/3' s activity 
le non-trivial, but ep-sy to follow, is before, an arrow 
indicates the point of attention defined by the pointer in 
the first problem on the problem list. 
The first test selected vas T1. 
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Tim cull.- to ?C32&. Ici't t&o i"ollov:lr.^: 
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The obvious action here.was to prune out the T2 and bud eone thing 
In to separate G and K. Only T5 tad T3 could separate then 
so T5 t the cheaper, W&e used. 



5VL 




3ut at this point EXCEAEGE saie 3 cistalce because of an obscure 

* 
bug and switched Tl and T5* 

T4 







II 



v:;Jl -"-.; ~"3 irare oai'cfi nr.a thd problem rsaools.tcd vltl- tha 
*4 -.orlc sae r,ovoi ftra-i the rocr.ll list tc -;::c jwebloa lirt. 
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SUBSTITUTE then noticed that 22 yielded a perfect quarteriac al- 
though slightly nore expensive than T4. 
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:'ow a 2CD split D and H with T* and was followed inaediately 
by EXCXASGB switching T5 and Tl. 
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11- Tj I-.;.a taa advaatc-SO 01" covliC tV.o core cDEtly tcct uo*.;.-. 
the tree. Afser a luB, the top node Was r-.-jaia the ce:i«er or 
r-ttcntior.. 
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Siace Cj^^ Cj 2 , sweep vss clearly in order, 
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At that point, T2 was replaced by the Shorter S4 
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Anti-cycle protection excluded any activity at the top. fur- 
ther operations involved only pruning out dead tests aa3 separatln; 
A and B for a fiiuil average cost of 3.79 zci e tree: 




fci fc* is] tai 



Id 



te) hi 



This tree seens reasonable in nany respects. ?1 is the least 
costly test that exactly divides the probability. T2 r-ud 
T4 arc also relatively short and together with Tl exactly 
quarter the probability. The longer tests are found only in the 
last row, where , with one exoeptioa, they are absolutely requires 
to complete the separation. ? h e exception is that 14 would be 



4\- 

fc^ter *«■ nciswr&tldfl - cjUI ^ t;.-:r. »culi 25. Z\c :-j*:-o.- lor --V- 
e^ycr i- t;;at ;;,1: i«:<v.-ico o> E$ was ariglai-ily iK.&«&Ilt.& 
to ec2«iratc J and Ii» tit tji?aii£jh a acrit,a «£ tracs-f^r— aiion?- 
rosultin; from 3S31iA3G3| S-fSBP* tLud Swi5*lB0t?J| It fou:- iteoif 
wi^b an entirely different duty. If do unsuitable a test vcr& 
x:- occur higher ia the tree, tfinssiTUSci or another operation 
ni^ht veil frave excise* it, but such transform are swallow 
at tba tin3 of the tree where there are no taste ttllowlttS* 
ft special case transformation could easily be written to handle 
this sort of situation, 

A Siccer rrobles 

In another experlnent, twice as many objects were involved 
and the number of tests was exi>aaded from 5 to 7, ?ha 
parameters involved are shown in table II* In this case, 
however, the outcomes were set randomly Kith the aid or a coin, 
and the costs and probabilities similarly fixed With the aid of 
a Boston telephone book. 

On this data the program rather uneventfully Generated 
the tree of figure 22* It is instructive, however, to notice 
how the tests and objects are deployed: 

Tests 4, 3, 7 nnd 5 split the objects into sets of nearly 
equal number. Moreover, 4, 3, and 7 cost less than any others, 
2hat they populate the highest regions of -he tree is therefore 
not unexpected. 

The expensive test 5 is found only in ona place, and reference 
to the table c* outcomes shov:s no other tsst can function as it 
does to separate M and E. 

iTote that tvo sets rasain ur.se;:arated: (k ?] and 
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Table II 
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?isure 22. Trad erocied for bigger problac. 
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. C;c CEpoeta to rir,5 tl:c r.orc ^rcix/Jlc tcr,ts Iil^hcr 1:: t«i 
tree v/.arc fever teats cro required to identify the;;. Sba r,os^ 
l*rob,-iblc objects are J, ? f and 2. Kotico t^ey are found ia l-io 
lil^;;est populated lavol. 






■ i 



Che pro^rfia as It a tanas did co_jc Ii:ttll; t ;eat thin^r; ;:li;. 
- niabex of situations to ttiich it Kac addressed. On others, 
serious nista;;cc were tiado, but the overall performance tcenod 
acceptable as a first try. Itu ouccec3C3 indioate value in the 
heuristic approach, while itis failures reveal new problems 
to be explored. 

oflaeaent 

Alterations to the present program would fall in throe 
catesories: bug removal, tuning* and major revision. Sur 
rescvql ourrently refers to the two faults observed in the 
5Z5jI73 section: first, a subroutine is needed to insure that 
the tests at the tips of the tree are optlBlffli second, EX02I2GE 
blunders frequently and swaps tests when it should not. Cor- 
rection of these weaknesses should involve no major difficulty.* 

Tuning refers to improvement of the simple criteria the 
program uses to make i"s decisions. ?or example, there may 
be better ways to define the priority number or to determine 
the Quality of a test from its cost and splitting ability, is 
was mentioned, cany of these thinss were set somewhat arbitrar- 
ily. 

?inally, nsjor revision refers to substantial changes which 

would Give the program aore of the qualities characteristic of 

"intelligence." ?or one thing, the prograa now has no facility 

Sox retreating from blind alleys; the condition for change is 

"change if tests indicate that it looks lii:e a good idaa" 

instead of "change if -.he res ultant structure does in fact 
% See Appendix II 
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Lc - to ctt^r results." 

:, *cco.:*; unjor revision would involve rcvtoias PC^riJ. 
As it fttesie, the ftuwtio,; Just c.-cckc to aoc If any trr-azfcr- 
Eitlttss can m used and bliivily uses tac first th*t tforia* 
One ciltcraatlve uould try all passible ch^nsoo and use the 
on* that vox£> "bost." Efcio, however, Is ^ fora of oahsustlvfl 
0tttt3*ratlpa. It vould be acre elegant to Introduce a function 
thnt coac^o;; vould auickly examine tree structure in the vloin- 
ity, fora an opinion on what it noad3, and then check to see if 2 
transformation appropriate to correction can be used. 

Generalisation 



It is easy to think of roneraliaed problems on vhich extend- 
ed prograas could be used, A few are listed below: 

1. Eie test ouxcoaes are not United to just two possibilities 

2. The objects are partitioned into catesories. 2he tester 
is concerned only with determining the category. 

3. 3Ve test outcomes are probabilistic functions of 

the object, 

■ 

A* Certain tests are inappropriate unless preceded oy 
certain others. 

5. Certain important objects auat not require core than 
a certain cost before recognition. {£ Hon appearing 
before the reader is unlikely, but he should be glad 
to recognize it quickly). 



The SoiiK&lil-Solttnd nCmOd 



fee ^athod 



^I-ice the 3alaw&ld-3olAfld scheme Is well presented ia their 
paper,* * only the £lat of the idea will be revitncGd here. 

Suppose, "Ivan a partially constructed tree, It is pos- 
sible to establish a lower bound on the expected cost of any 
complete tree formed by budding out the tree at hand- lot this 
lower bound to the expected cost be L- where T is the iiaae of 
the associated tree. 

Then if a group of partial trees are candidates for the 
trunk of the final tree, it would be wise to try budding out 
the candidate whose L„ is smallest until the L_ of the derived 
tree exceeds that of sono other candidate on the original list. 

This general idea led to the flow chart of figure 23. The 
original set of partial trees consists of all trees contaiain£ 
just one test. Tfreee are ordered according to their asso- 
ciated lover bounds, the Lj l s. The first member of the list 
is picked off and all trees that can be forced from it by a 
single test addition are plaoed on the partial tree list in 
the appropriate plaoe. Eie algorithm terminates when a com- 
plete tree appears at the head of the list- This tree is optical 
because its 1* is not only a lower bound, but ie in fact the 
expected cost; and that cost ust certainly be lower than that 
of any complete tree that evolves from partial trees vhosa iVs 
are greater. t 

::ow suppose we thinJc of the S tests as foaming 3 coordinates 
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fori all possible trees 
of depth one awl place 
then on the partial tree 
list in order of increas- 
ing I T 



I 



> 



pick off first 
aember of partial 
tree list 




xorn all partial treas 




possible by the addition 




of or,ly one test — 




place these trees on the 




pirtial tree list iu 




the spot dictated by 




their 1 T 
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?l S ure 23- ?n& Seinwald-Soland AZgpsltaa. 
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of a i.;-co. Slisa each object lies oil so:.w oorscr a." iijc iti-i-jlt-nt 
;: iirwcsiocal cube. A thin ripnoa will lo aefi&etl i.-iTo-.-..:/ 
ao o:ic In which few if :.:y objects lie on ci'noost ccr_or5 -cr 
which only one coordinate differs." 

;y contention is that for reaaoaab]/ lar^c , thin Dpaoep, 
the ?.ei'iv;a.ld-2olaiid anthod amounts essentially to exhaustive 
enumeration of possible trees. 

DrTgr.erttlon of the Method 

A study of the paper reveals that is thin spaces the rule 
for forming L-, simplifies to: 

1) find the expected cost of each te3t by multiplying 
its cost by the totaJ probability of the objects that 
course through it. 

2) sua the results. 

?hus it is clear that each tine a coaplete layer is added,- the 
lower bound is increased 'oy the average costs of the tests 
in the layer. The lower bound, L n , therefore Increases 
steadily with the depth of q partial tree. 

Tae reason for the failure of the method in thin spaces 
lies in this steady growth of L- with tree depth; if any 
tree of a certain depth has been examined, it is likely that 
all trees with two or three fewer layers are on the partial tree 
list. :-;ence, before any complete tree can be formed, an enormous 
nurber of partial trees of only slightly less depth have been 
enuaer&ted, evaluated, and placed on the partial tree list. 3ut 
it has been shown that enumeration of all trees of any con-trivial 



* Co-munlcations enthusiasts prefer to say that tha siniaaa 
Haaadxg disxar.ee separatism objects is 2. 
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-Icpth is out of the question* 

ATPENDIX II 

Bug s 

The bugs mentioned on pages 27 and 31 have been somewhat allirt.t«l 
since tMs paper was completed in 1967, 

Blunder, by the EXCHANGE Juristic are prevented by blocking action 
if substantial rerouting take* place. Befer to figure U, page 13. 
EXCHANGE does not ace unless 

P( ^part ofAj ) > K P(A). 

>. ia currently .8 . 

A function wiled mat e^tnate, the probl« f inappropriate 
t«t fi at the ttee'e tips. It .imply checks to ».e if „ y other tost 
can do the iW final splitting job with lower cost. The FORERAN 
machinery applies HEWTtP whenever a tip is encountered. 
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